Skip to content

Dispatch extraction on header constants instead of comparing names - #13613

Draft
dougqh wants to merge 5 commits into
dougqh/kafka-carrier-cachefrom
dougqh/header-schema
Draft

dougqh wants to merge 5 commits into
dougqh/kafka-carrier-cachefrom
dougqh/header-schema

Conversation

@dougqh

@dougqh dougqh commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

What Does This Do

Adds HeaderSchema, an enum of the headers the tracer knows by name, and dispatches extraction on those constants instead of comparing header names. Extracted contexts are unchanged.

Stacked on #13606.

Motivation

A 30-minute CPU profile of a Kafka consumer had ~6% of the consumer thread's tracer time in header-name comparisons: each record's headers were compared, case-insensitively, against several names by the span id cache, the baggage extractor and the Datadog and W3C interpreters, about 30 equalsIgnoreCase calls per record. #13605 already matches each key once against the propagators' declared names; this carries the result of that match along as a constant, so later code switches on it.

Additional Notes

  • HeaderSchema (components/context): Datadog, W3C, baggage, B3, Haystack, X-Ray and DSM headers, plus the forwarding / user agent / client IP headers every style reads. Each constant has its lower-case name and ASCII bytes. Its ordinal() is a dense index: carrier-specific forms of a name (a Netty AsciiString, a gRPC Metadata.Key) belong in arrays indexed by ordinal next to their instrumentations, which keeps the enum dependency-free.
  • CarrierKeys records the constant of every declared name that is a known header, and returns it from header(String) / headerAscii(ByteBuffer) (with an identity shortcut, since carriers that matched a key's bytes pass the declared name itself).
  • The composite propagator's cache records each pair's constant (IndexedCarrier.header).
  • KeyClassifier.accept(HeaderSchema, key, value), defaulting to the by-name accept. ContextInterpreter implements it: a style's own headers go straight to its handling, the shared forwarding and user agent headers to their handlers, and other known headers to the common tail (client IP, header tags, mapped baggage). A null constant, header-name logging, or a style that doesn't classify the header takes the by-name path, so the two paths can't disagree.
  • Datadog, W3C and Haystack classify their headers (x-amzn-trace-id, read only with AWS propagation, stays by name); None, with no headers of its own, sends every known header to the shared handling. B3 and X-Ray stay by name: their accept wraps the shared handling and the common tail in a try/catch, which the generic dispatch would move outside it.
  • ExtractionCache's span id check, the single-extractor path and the baggage extractor use the constants too.
  • Tests: HeaderSchemaTest; CarrierKeysTest / CarrierCacheTest for constants; ExtractorKeysForkedTest checks, for every style (with its own keys and with every known header) and for the compound Datadog + W3C extractor reading the carrier in place, that dispatching on constants extracts the same context as by name. Propagation and baggage tests pass unchanged (1,431, including the compound-extraction golden test, which goes through the composite propagator and so through the constant path).
  • The carrier cache stores each pair's constant as a byte (ordinal() + 1, 0 for none), not a second reference array, keeping the per-extraction cost to a byte[8].
  • Numbers: neutral on CPU, so the case for this PR is structural.

🤖 Generated with Claude Code

@dougqh dougqh added comp: core Tracer core tag: performance Performance related changes type: refactoring tag: ai generated Largely based on code generated by an AI or LLM labels Nov 21, 2026
@datadog-official

datadog-official Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

🎯 Code Coverage (details)
• Patch Coverage: 84.62%
• Overall Coverage: 59.60% (+0.04%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 93a92d8 | Docs | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.08 s 13.99 s [+0.0%; +1.3%] (maybe worse)
startup:insecure-bank:tracing:Agent 13.04 s 13.09 s [-1.0%; +0.4%] (no difference)
startup:petclinic:appsec:Agent 17.10 s 17.00 s [-0.3%; +1.5%] (no difference)
startup:petclinic:iast:Agent 16.79 s 17.10 s [-2.7%; -0.9%] (maybe better)
startup:petclinic:profiling:Agent 16.14 s 16.49 s [-6.2%; +1.9%] (no difference)
startup:petclinic:sca:Agent 17.13 s 16.99 s [+0.0%; +1.7%] (maybe worse)
startup:petclinic:tracing:Agent 16.20 s 15.85 s [-2.1%; +6.6%] (no difference)

Commit: 93a92d87 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@pr-commenter

pr-commenter Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

Kafka / producer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch dougqh/kafka-carrier-cache dougqh/header-schema
git_commit_date 1795289307 1795289464
git_commit_sha 5bc558d 93a92d8
See matching parameters
Baseline Candidate
ci_job_date 1795290966 1795290966
ci_job_id 2286652785 2286652785
ci_pipeline_id 155480150 155480150
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
jdkVersion 11.0.31 11.0.31
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=15002 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=15002 -Dhttp.nonProxyHosts=localhost *.localhost
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-31eoknqh 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-31eoknqh 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu 11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaProduceBenchmark.benchProduce same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaProduceBenchmark.benchProduce same

@pr-commenter

pr-commenter Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

Kafka / consumer-benchmark

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch dougqh/kafka-carrier-cache dougqh/header-schema
git_commit_date 1795289307 1795289464
git_commit_sha 5bc558d 93a92d8
See matching parameters
Baseline Candidate
ci_job_date 1795291206 1795291206
ci_job_id 2286652787 2286652787
ci_pipeline_id 155480150 155480150
cpu_model Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz
jdkVersion 11.0.31 11.0.31
jmhVersion 1.36 1.36
jvm /usr/lib/jvm/java-11-openjdk-amd64/bin/java /usr/lib/jvm/java-11-openjdk-amd64/bin/java
jvmArgs -Dhttp.proxyHost=127.0.0.1 -Dhttp.proxyPort=15002 -Dhttps.proxyHost=127.0.0.1 -Dhttps.proxyPort=15002 -Dhttp.nonProxyHosts=localhost *.localhost
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-j72rgn1r 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-j72rgn1r 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
vmName OpenJDK 64-Bit Server VM OpenJDK 64-Bit Server VM
vmVersion 11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu 11.0.31+11-post-1ubuntu1-22.04.2-Ubuntu

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 3 metrics, 0 unstable metrics.

See unchanged results
scenario Δ mean throughput
scenario:not-instrumented/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-disabled-benchmarks/KafkaConsumerBenchmark.benchConsume same
scenario:only-tracing-dsm-enabled-benchmarks/KafkaConsumerBenchmark.benchConsume same

dougqh and others added 5 commits November 21, 2026 14:31
An enum of the carrier keys propagators read (Datadog, W3C, baggage, B3,
Haystack, X-Ray, DSM, and the forwarding / user agent / client IP headers
every style reads), each with its lower-case name and ASCII bytes. Its
ordinal is a dense index for per-header arrays, which keeps carrier-specific
forms (a Netty AsciiString, a gRPC Metadata.Key) next to their
instrumentations.

CarrierKeys records the constant of every declared name that is a known
header and returns it from header(String) / headerAscii(ByteBuffer), so
code handling several headers can switch on it instead of comparing names.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The composite propagator's cache records each key's HeaderSchema constant
(IndexedCarrier.header), and passes it along:

- KeyClassifier gains accept(HeaderSchema, key, value), defaulting to the
  by-name accept.
- ContextInterpreter's implementation sends a style's own headers straight
  to its handling, the shared forwarding / user agent headers to their
  handlers, and other known headers to the common tail (client IP, header
  tags, mapped baggage). A null constant, header-name logging, or a style
  that doesn't classify the header takes the by-name path.
- The Datadog and W3C interpreters classify their headers this way; the
  other styles keep the by-name path for now.
- ExtractionCache's span id check, the single-extractor path and the
  baggage extractor use the constants too.

A profile of a Kafka consumer had ~6% of the consumer thread's tracer time
in these name comparisons (about 30 equalsIgnoreCase calls per record).
ExtractorKeysForkedTest checks that, for every style and for the compound
extractor, dispatching on constants extracts the same context as by name.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The carrier cache kept a HeaderSchema[] beside its pairs, a second
reference array per extraction (about 48 bytes for the usual 8 pairs).
With fewer than 127 constants, ordinal() + 1 fits in a byte, so it's a
byte[] instead (24 bytes).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Haystack classifies its trace / span / parent id headers like Datadog and
W3C; None has no headers of its own, so every known header gets the shared
handling directly. B3 and X-Ray stay by name: their accept wraps the shared
handling and the common tail in its try/catch, which the generic dispatch
would move outside it. ExtractorKeysForkedTest's per-style equivalence
check covers the two new styles.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Same as CarrierKeys.contains: keys are usually lower case already, so
String.equals (intrinsified, with an identity check) catches them before
the case-insensitive comparison.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dougqh
dougqh force-pushed the dougqh/header-schema branch from d5708a4 to 93a92d8 Compare November 21, 2026 19:39
@dougqh
dougqh force-pushed the dougqh/kafka-carrier-cache branch from 8131802 to 5bc558d Compare November 21, 2026 19:39

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: core Tracer core tag: ai generated Largely based on code generated by an AI or LLM tag: performance Performance related changes type: refactoring

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant